Skip to content

Update example_cloud_run system test#40540

Merged
eladkal merged 1 commit into
apache:mainfrom
VladaZakharova:update-example-cloud-run-test
Jul 2, 2024
Merged

Update example_cloud_run system test#40540
eladkal merged 1 commit into
apache:mainfrom
VladaZakharova:update-example-cloud-run-test

Conversation

@e-galan

@e-galan e-galan commented Jul 2, 2024

Copy link
Copy Markdown
Contributor

Add the clear_args parameter to a Cloud Run job request used in the example_cloud_run dag.

Although the documentation states the the clear_args parameter in google.cloud.run.v2.RunJobRequest.Overrides.ContainerOverride object is optional, in rare cases its absence can cause an error when creating a RunJobRequest

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/google/cloud/operators/cloud_run.py", line 298, in execute
    self.operation = hook.execute_job(
                     ^^^^^^^^^^^^^^^^^
  File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/google/common/hooks/base_google.py", line 524, in inner_wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python3.11/lib/python3.11/site-packages/airflow/providers/google/cloud/hooks/cloud_run.py", line 122, in execute_job
    run_job_request = RunJobRequest(
                      ^^^^^^^^^^^^^^
  File "/opt/python3.11/lib/python3.11/site-packages/proto/message.py", line 609, in __init__
    pb_value = marshal.to_proto(pb_type, value)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python3.11/lib/python3.11/site-packages/proto/marshal/marshal.py", line 228, in to_proto
    pb_value = self.get_rule(proto_type=proto_type).to_proto(value)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python3.11/lib/python3.11/site-packages/proto/marshal/rules/message.py", line 36, in to_proto
    return self._descriptor(**value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Protocol message ContainerOverride has no "clearArgs" field.

making the example_cloud_run dag flaky.

Adding the parameter should fix that.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@boring-cyborg boring-cyborg Bot added area:providers area:system-tests provider:google Google (including GCP) related issues labels Jul 2, 2024
@eladkal eladkal merged commit 09bbc9c into apache:main Jul 2, 2024
romsharon98 pushed a commit to romsharon98/airflow that referenced this pull request Jul 26, 2024
@e-galan @romsharon98
Update example_cloud_run system test (apache#40540)
401171a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@eladkal eladkal eladkal approved these changes

Assignees

No one assigned

Labels

area:providers area:system-tests provider:google Google (including GCP) related issues

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@e-galan @eladkal